Bifurcated conferencing functions

ABSTRACT

A source conference function is provided for each source device, and a destination conference function is provided for each destination device. Any given user terminal may act as both a destination device when receiving media content and a source device when sending media content to facilitate bidirectional conferencing. The source device generates media content, which is sent to the associated source conference function. In general, the source conference function identifies destination conference functions to which the media content should be delivered, and delivers the media content to the identified destination conference functions. Any given destination conference function receives the media content from this and other source conference functions. The media content received from all or select of the source conference functions may be processed by the destination conference function as desired, and then mixed together to form destination media content, which is delivered to the destination device.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 11/616,701, which was filed on Dec. 27, 2006, the disclosure ofwhich is hereby incorporated by reference in its entirety.

FIELD OF THE INVENTION

The present invention relates to media conferencing, and in particularto bifurcating conferencing functionality into source and destinationconference functions that are associated with conference participants.

BACKGROUND OF THE INVENTION

Conference bridges are used to facilitate conference calls between twoor more conference participants. In operation, multiple calls frommultiple communication terminals are terminated at a conference bridge.In simpler conference bridges, the audio signals from each of thecommunication terminals are mixed together and provided back to each ofthe communication terminals. In more advanced conference bridges, theaudio levels for the audio signals from the different communicationdevices are monitored and used to identify one or more of the audiosignals to provide as an output to the communication devices. Ingeneral, the communication terminal or terminals providing the audiosignals having the highest level at any given time are selected as theselected communication terminal. The audio signals from the selectedcommunication terminals are mixed together and delivered all of thecommunication terminals in the conference. The audio signals from theunselected communication terminals are dropped, and thus, are notprovided to the communication terminals. As such, only the audio signalsprovided by the selected communication terminals are presented to thecommunication terminals in the conference. At any given time, none ofthe participants will hear any participants other than those using theselected communication terminals. To avoid distractions, the audiosignal from a selected communication terminal is generally not providedback to itself.

Since conference bridges generally select the audio signals from theloudest participant to present to other participants, there are manysituations where other participants are contending for conference timeto no avail. If those participants contending for conference time do notpresent audio signals at a level higher than the participant using thecurrently selected communication terminal, they will never be selectedas the selected participant. This situation is problematic forsoft-spoken participants as well as participants in a conference with arelatively active and loud participant. In many instances, a louderparticipant may gain conference access and maintain conference accessfor extended periods of time, even though other participants arecontending for conference access. Generally, the other participants havelittle control over the conference itself, and have little or noinfluence on prioritizing themselves or others in gaining conferencecontrol.

Further, conference calls are being employed in more diverseapplications. For instance, on-line gaming groups are employingconference calls to allow multiple garners from different locations totalk with each other during a gaming session. Given the excitement andpotential for significant background music or other noise, those garnerswith the louder group or environment may dominate the conference simplydue to the selection process used by the conference bridge hosting theconference call. Again, participants have little control over theirconference experience and the louder participants may gain conferenceaccess and maintain conferences access for extended periods of time,even though other participants may want or need conference access.

Further, audio conferencing for these gaming environments are oftenunrealistic relative to the virtual environment of the game. Within agame environment, there are few scenarios where all of the participantsshould be able to communicate with each other all of the time. Ideally,there would be multiple conferences for participants on individual teamand yet another conference for all of the participants, regardless ofteam affiliation. However, existing conference bridges are unable tosupport multiple conferences for a participant or group of participantsat the same time. Further, the centralized nature of conference bridgesmakes integrating the gaming controller with the conference bridgepractically impossible. Thus, the sounds of the game are generallyseparated from the sounds of the participants.

Gaming environments as well as traditional conference environments aresupporting more and more participants in a given conference. As thenumber of participants grows, network resources are being taxed. Thecentralized nature of the conference bridges limits the scalability ofthe conferencing. To increase the number of participants a conferencebridge can support, additional ports and processing resources are addedto the existing conference bridge, or a new conference bridge is added.

Accordingly, there is a need for a scalable conferencing mechanism thatimparts greater conference control to the individual participants. Thereis a further need for a conferencing mechanism to support multipleconferences for a given participant or group of participants as well asprovide a platform that can be integrated with other systems, such asgaming systems.

SUMMARY OF THE INVENTION

The present invention bifurcates traditional media conferencingfunctionality into source and destination conference functions to createa flexible and readily scalable media conferencing environment.Bifurcating media conferencing in this manner can conserve networkresources as well as support applications that were previouslyimpracticable or impossible. A source conference function is providedfor each source device, and a destination conference function isprovided for each destination device. Any given user terminal may act asboth a destination device when receiving media content and a sourcedevice when sending media content to facilitate bidirectionalconferencing. The source device generates media content, which is sentto the associated source conference function. In general, the sourceconference function identifies destination conference functions to whichthe media content should be delivered, and delivers the media content tothe identified destination conference functions. Any given destinationconference function receives the media content from this and othersource conference functions. The media content received from all orselect of the source conference functions may be processed by thedestination conference function as desired, and then mixed together toform destination media content, which is delivered to the destinationdevice.

Media content is generally delivered in media streams. A destinationconference function may receive from multiple source conferencefunctions any number of media streams associated with a givenconference. The destination conference function may select one or moreof the media streams as active media streams based on certain controlinformation. The control information may be provided by any entity orfunction as well as be derived from the media content itself. Forexample, the destination conference function may analyze the audio-basedmedia content from the various source conference functions to determinerelative volume levels of the respective sources, wherein the mediastreams associated with the highest volume levels are selected as theactive sources. In another example, the control information is relatedto a priority to associate with a given media stream, or informationhelpful in determining whether to select the media stream as an activemedia stream, and how to process the media stream if it is selected asan active media stream. In the latter case, orientation information maybe provided as control information to identify a participant's actual orvirtual location, direction, or a combination of thereof to aid inselecting active media streams and processing the active media streams.Such orientation information may be absolute or relative to that ofother participants.

Various types of control information may be provided to the respectivesource and destination conference functions to control any givenconference. In additional to receiving information from a centralizedcontrol system, the source and destination conference functions mayprovide control information to each other. The control information maybe exchanged using in-band or out-of-band signaling. In-band signalingentails providing the control information in the packets of the mediastream carrying the media content. Out-of-band signaling entailsdelivering the control information outside of the media stream carryingthe media content, and as such, may be delivered directly or via thecentralized control system.

The active media streams that are selected for a given destinationconference device may be processed based on the same or differentcontrol information and then mixed together to form the destinationstream, which carries the destination media content to the destinationdevice. Notably, the destination conference function may supportmultiple conferences at any given time for the destination device. Mediastreams associated with different conferences and selected for a givendestination device may be mixed together to form the destination stream.The destination media content in the destination stream may have contentfrom different conferences at the same time, such that a participantassociated with the destination device is presented a unique blend ofmedia content from the various conferences. Accordingly, eachparticipant may be presented a unique blend of media content fromvarious sources for one or more conferences at any given time.

In certain embodiments, a conference system may support multipledestination conference functions. If a source conference function mustsend media content for a given source device to multiple destinationconference functions that are provided in a given conference system, themedia content may be delivered to the multiple destination conferencefunctions using a single, multicast stream. As such, the use ofredundant unicast streams between the source conference function and thedestination conference functions is avoided.

Those skilled in the art will appreciate the scope of the presentinvention and realize additional aspects thereof after reading thefollowing detailed description of the preferred embodiments inassociation with the accompanying drawing figures.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawing figures incorporated in and forming a part ofthis specification illustrate several aspects of the invention, andtogether with the description serve to explain the principles of theinvention.

FIG. 1 is a logical representation of a conference environment accordingto one embodiment of the present invention.

FIG. 2 is an example of a conference facilitated in the conferenceenvironment illustrated in FIG. 1 according to one embodiment of thepresent invention.

FIG. 3 is a flow diagram illustrating operation of a source conferencefunction according to one embodiment of the present invention.

FIG. 4 is a flow diagram illustrating operation of a destinationconference function according to one embodiment of the presentinvention.

FIG. 5 is a logical representation of various source conferencefunctions and destination conference functions being supported by one ormore conference systems according to one embodiment of the presentinvention.

FIGS. 6A-6E are examples of different implementations of sourceconference functions, destination conference functions, and mixingfunctions according to several embodiments of the present invention.

FIG. 7 is a logical representation of a conference environment includingan alternate media source according to one embodiment of the presentinvention.

FIG. 8 is an example of a table that may be maintained at the source ordestination conference functions to keep track of the participantsassociated with each conference according to one embodiment of thepresent invention.

FIG. 9 is a logical representation of a virtual conference environmentaccording to one embodiment of the present invention.

FIG. 10 is an example of a table that may be provided to a destinationconference function according to one embodiment of the presentinvention.

FIG. 11 is a block representation of a conference system according toone embodiment of the present invention.

FIG. 12 is a block representation of a user terminal according to oneembodiment of the present invention.

FIG. 13 is a block representation of a control node according to oneembodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The embodiments set forth below represent the necessary information toenable those skilled in the art to practice the invention and illustratethe best mode of practicing the invention. Upon reading the followingdescription in light of the accompanying drawing figures, those skilledin the art will understand the concepts of the invention and willrecognize applications of these concepts not particularly addressedherein. It should be understood that these concepts and applicationsfall within the scope of the disclosure and the accompanying claims.

With reference to FIG. 1, a logical representation of a conferenceenvironment 10 is provided according to one embodiment of the presentinvention. In any conference environment, any number of user terminalsis able to participate in a conference such that the participants areable to communicate with each other. At any given time, certain userterminals are generating media content that is being presented to otheruser terminals that are engaged in the conference call. During thecourse of the conference, each user terminal may be a source for mediacontent as well as a destination for media content, generally dependingon whether the associated participant is talking or listening. As such,user terminals may act as a content source or a content destinationduring the conference. For clarity in the following description, theuser terminals are represented as either source devices 12 ordestination devices 14, depending on whether they are acting as acontent source or a content destination. A source device 12 and adestination device 14 may represent a single user terminal or differentuser terminals. The source devices 12, destination devices 14, andvarious other functions are often further referenced with a letter inparenthesis, such as 12(C). These references are used to specificallyidentify a given entity for particular examples, such as source device12(C). The user terminals may take any form, such as personal computers,personal digital assistants, mobile telephones, or like communicationdevices. The communications sessions facilitating the conferencefunctions may extend over wired or wireless circuit switched or packetnetworks. As with the associated conferences, the communicationssessions may be established as needed or maintained perpetually.

The present invention bifurcates traditional media conferencingfunctionality into source and destination conference functions to createa flexible and readily scalable conferencing environment 10. Bifurcatingmedia conferencing in this manner can conserve network resources as wellas support applications that were previously impracticable orimpossible. A source conference function 16 is provided for each sourcedevice 12, and a destination conference function 18 is provided for eachdestination device 14. As noted, any given user terminal may act as botha destination device 14 when receiving media content and a source device12 when sending media content to facilitate unidirectional orbidirectional conferencing. The source device 12 generates mediacontent, which is sent to the associated source conference function 16.In general, the source conference function 16 identifies destinationconference functions 18 to which the media content should be deliveredand delivers the media content to the identified destination conferencefunctions 18. Typically, for every user terminal there will be a sourceconference function 16 and a destination conference function 18.However, this is not necessary in each application as a user terminalmay have either a source conference function 16 or a destinationconference function 18.

As illustrated in FIG. 1, the media content generated from the sourcedevice 12(C) is provided to the source conference function 16(C), whichwill identify the destination conference functions 18 to which the mediacontent from the source device 12(C) should be sent. Assuming the mediacontent should be sent to destination conference functions 18(A), 18(B),and 18(N), the source conference function 16(C) will deliver the mediacontent to the destination conference functions 18(A), 18(B), and 18(N).These destination conference functions 18 will then process the mediacontent to further conference functionality. Notably, each of the othersource conference functions 16(A), 16(B), and 16(N) may be operating ina similar fashion for the same or different conferences, where therespective media content from the corresponding source devices 12(A),12(B), and 12(N) is sent to the same or different destination conferencefunctions 18.

As such, any given destination conference function 18 may receive themedia content from any number of source conference functions 16 inassociation with one or more conferences. The media content receivedfrom all or select source conference functions 16 may be processed bythe destination conference function 18 as desired and then mixedtogether by a mixing function 20 to form destination media content,which is delivered to an associated destination device 14. Asillustrated in FIG. 2, destination conference function 18(C) receivesthe media content from source conference functions 16(A), 16(B), and16(N) in association with one or more conferences. The media contentreceived from the source conference functions 16(A), 16(B), and 16(N)may be processed by the destination conference function 18(C) as desiredand then mixed together by the mixing function 20 to form destinationmedia content, which is delivered to destination device 14(C).

Media content is generally delivered in media streams. A destinationconference function 18 may receive from multiple source conferencefunctions 16 any number of corresponding media streams for a givenconference. The destination conference function 18 may select one ormore of the media streams as active media streams based on certaincontrol information. The active media streams are then mixed together bythe mixing function 20 to generate the destination stream, which isdelivered to the corresponding destination device 14. The unselectedmedia streams are left out of the destination stream. Again, thedestination stream may be formed from active media streams associatedwith multiple conferences in which the participant associated with thedestination device 14 is participating.

Notably, the term “active” is used only to identify a selected mediastream and is not necessarily indicative of the presence or absence ofcontent on selected or unselected media streams. Unselected mediastreams may have various levels or types of content and may not beselected for various reasons. In other words, the term “active” is notdescribing the actual media content, but is merely used to indicatewhich media stream has been selected.

The active media streams that are selected for a given destinationdevice 14 may also be processed based on the control information for thesame or different conferences before being mixed together to form thedestination stream. Different active streams may be uniquely processedbased on unique control information prior to mixing. As such, thedestination media content in the destination stream may have content fordifferent conferences at the same time, such that a participantassociated with the destination device 14 is presented a unique blend ofmedia content from the various conferences. Those skilled in the artwill recognize that actual stream processing and mixing may beimplemented in any number of ways, and in particular, may be implementedsequentially or concurrently.

The control information may be provided by any entity or function aswell as derived from the media content itself. As illustrated in FIGS. 1and 2, a logical control plane 22 is provided between the source anddestination conference functions 16 and 18. The control plane 22represents a control path to allow the source and destination conferencefunctions 16 and 18 to receive control information from each other orfrom another entity or node, such as a control node 24. Although thecontrol plane 22 is shown apart from the media streams that carry themedia content, the control plane 22 may be implemented within the mediastreams, outside of these media streams, or a combination thereof. Assuch, control information may be derived from the media content or thesource and destination conference functions 16, 18 may provide specificcontrol information to each other using in-band or out-of-bandsignaling. In-band signaling entails providing the control informationin the packets of the media stream carrying the media content.Out-of-band signaling entails delivering the control information outsideof the media stream carrying the media content, and as such, may bedelivered directly or via the control plane 22, as well as by thecontrol node 24.

For example, a destination conference function 18 may analyzeaudio-based media content received from the various source conferencefunctions 16 to determine relative volume levels of the respectivesources, wherein the media streams associated with the highest volumelevels are selected as the active media streams. In this instance, thecontrol information is derived from the media content. Alternatively,the source destination function 16 may analyze energy, power, or levelcharacteristics of a media stream from a corresponding source device 12to systematically identify volume level metrics for the media content.These metrics represent control information, which are systematicallyinserted as such in all or certain packets of the media stream that issent to the various destination conference functions 18. The metrics mayalso be sent in a separate message or in separate signaling inassociation with the media stream to the destination conferencefunctions 18, which will use the control information accordingly.

In another example, the control information is related to a priority toassociate with a given media stream, or other information helpful indetermining whether to select the media stream as an active mediastream. Again, the control information may also provide instructions forprocessing the media stream when it is selected as an active mediastream. Orientation information may be provided as control informationto identify a participant's actual or virtual location, direction, or acombination of thereof to aid in selecting active media streams,processing the active media streams, or both. Such orientationinformation may be absolute or relative to that of other participants.

In the following flow diagrams, various types of control information aredescribed. Although multiple types of control information are used toillustrate an example of how the invention may be employed, theparticular control information for any given embodiment need not be ascomprehensive as that provided in the illustrated embodiment. In otherwords, although multiple types of control information are used in theillustrated embodiment, only a single type of control information isgenerally necessary to facilitate operation of the present invention.The various types of control information used in the following examplesinclude participant information, source priority information, sourceorientation information, acoustic properties, destination priorityinformation, destination orientation information, and supplemental mediainformation.

The participant information identifies participants or their associatedfunctions or devices for a given conference. With the present invention,a given participant may participate concurrently in multipleconferences. The source priority information is information accessibleby the source conference function 16, and bears on the relative prioritythat should be provided to the associated participant when determiningactive media streams. The source priority information may be a relativelevel metric determined from the media content received from the sourcedevice 12, or may be predetermined information that is unrelated to themedia content received from the source device 12. The source orientationinformation may be information bearing on the actual or virtual positionof the participant in an actual or virtual conference environment. Theposition information may bear on actual or relative location, direction,proximity, and the like within the conference environment or withrespect to other participants in the conference environment. Thesupplemental media information may relate to external media to providein association with the particular media stream or any destinationstream in which the media stream is provided. The acoustic properties,as described above, may include the actual or relative level metrics ofthe actual media content.

The destination priority information is similar to the source priorityinformation, except that it is provided by the destination participantinstead of the source participant. Accordingly, the destinationparticipant may assign a relative priority to associate with the sourceparticipant with respect to the destination participant, as well as withrespect to other source participants. The destination orientationinformation is similar to the source orientation information, exceptthat it is provided by a function or entity associated with thedestination participant. The actual information may relate to theorientation of the source participant, the destination participant, orany other participants in the actual or virtual conference environment.Again, these incarnations of control information are simply provided forthe purposes of illustration, and the present invention is not limitedthereto.

Turning now to FIG. 3, a flow diagram is provided to illustrateoperation of a source conference function 16 according to one embodimentof the present invention. Initially, the source conference function 16will obtain participant information for any number of conferences inwhich the participant associated with the source conference function 16is participating (step 100). The source conference function 16 will thenidentify the destination conference functions that should receive themedia stream of the source device 12 based on the participantinformation (step 102). Regardless of the number of conferences in whichthe source participant is participating, the media content from thesource participant is sent to all destination conference functions 18that are associated with participants in one or more conferences.

The source conference function 16 may then obtain source priorityinformation (step 104). The source priority information may be differentfor each of the concurrent conferences, and may bear on the relativepriority that should be applied to the media content from the sourceparticipant by the destination conference functions 18 that will receivethe media content. Similarly, the source conference function 16 mayobtain source orientation information for each conference (step 106). Inone example, the source orientation information may identify thelocation and direction of the source participant in the variousconference environments. Again, this information may be absolute or maybe relative to other participants in the respective conferenceenvironments. Additional control information in the form of supplementalmedia information may be obtained for each conference (step 108). Thesupplemental media information may take various forms and may relate toassisting the various destination conference functions 18 in obtainingadditional media to present to the destination participant inassociation with the media content of the source participant. Forexample, the supplemental media information may include backgroundsounds or music, as well as identify graphics or video clips to presentin association with the media content when it is presented to thedestination participant in the destination media content.

All or a portion of the control information that is available to thesource conference function 16 may need to be delivered to thedestination conference functions 18 in association with the mediacontent. As such, the source conference function 16 may provide sourcemixing information based on the source priority information, orientationinformation, the supplemental media information, or any combinationthereof for delivery to the destination conference functions 18 (step110). Different source mixing information may be provided for eachconference. If the source participant and destination participant arecommon participants in different conferences, different source mixinginformation may be sent to the same destination conference function 18for the different conferences.

As discussed above, the source mixing information may be sent to theappropriate destination conference functions 18 via in-band orout-of-band techniques. For in-band delivery, the source mixinginformation is inserted into all or certain packets of the media streamsthat are sent toward the various destination conference functions 18(step 112). For out-of-band delivery, the source mixing information issent to the appropriate destination conference functions 18 via messagesor signaling outside of the media streams (step 114). This process maytake place once for each conference, or systematically throughout theconferences, depending on the type of control information and how thecontrol information is used by the source and destination conferencefunctions 16, 18. If control information is updated throughout theconference, all of the control information may be provided at thebeginning of the conference, wherein certain of the conferenceinformation is updated throughout the conference and sent to thecorresponding destination conference functions 8.

Turning now to FIG. 4, operation of a given destination conferencefunction 18 is illustrated according to one embodiment of the presentinvention. Initially, the destination conference function 18 will obtainany available source mixing information provided in association withincoming media streams (step 200). Again, these media streams mayoriginate from various source conference functions 16 and may beassociated with one or more conferences. In this example, assume thedestination conference function 18 is configured to process therespective media streams to determine acoustic properties, such asrelative level metrics, for each media stream (step 202). Thedestination conference function 18 may also obtain any accessibledestination priority information (step 204), destination orientationinformation (step 206), and any supplemental media information (step208). Based on the acoustic properties, destination priorityinformation, destination orientation information, source mixinginformation, or any combination thereof, the destination conferencefunction 18 will select active media streams to present to thedestination device 14 (step 210). The control information obtained fromthe media content, or source conference functions 16 or other entities,as well as the control information provided at the destinationconference function 18 may be the same or different for the respectiveconferences. However, the destination conference function 18 will takeall the various conferences into consideration when determining whichmedia streams to select and use to create the destination stream whichwill be sent to the destination device 14.

In addition to merely using this control information to select theactive media streams to use to create the destination stream, theselected active media streams may be processed prior to or during mixingin light of the control information. Notably, certain controlinformation may be used to select active media streams, while othercontrol information is used to process the respective media streams.Other control information may bear on both the selection and processingof the media streams. Notably, the present invention does not requirethat the active media streams be processed prior to or during mixing tocreate the destination stream.

In addition to media streams from the various source conferencefunctions 16, supplemental media content may be obtained from thedestination device 14, the source device 12, or other entity, which mayor may not be associated with a participant. For audio-basedconferencing, the supplemental media content may include sound effectsand the like that are mixed with the selected media streams to createthe destination stream.

Continuing with FIG. 4, once the various active media streams from thesource conference function 16 and any supplemental media streams areselected and processed, these media streams are mixed together toprovide the destination stream (step 214), which is effectivelydelivered to the corresponding destination device 14 (step 216). Thisprocess takes place in a distributed fashion at the various destinationconference functions 18 for the various destination devices 14.Accordingly, conference functionality is not only bifurcated, but thebifurcated parts are respectively distributed among the various sourceand destination devices 12, 14.

The various source conference functions 16 and destination conferencefunctions 18 may be supported by one or more conference systems 26, asillustrated in FIG. 5. As such, one or more centralized and networkbased conference systems 26 may provide the respective source anddestination functions 16, 18 for any number of source and destinationdevices 12, 14, which generally represent user terminals. When a sourceconference function 16 needs to send media content to a cluster ofdestination conference functions 18 that are supported by a givenconference system 26, the source conference function 16 may send asingle multicast stream including the media content to the conferencesystem 26, which will distribute the media content provided in themulticast stream to the respective destination conference functions 18that need to receive the media content. As such, only a single mediastream is required between the source function 16 and the conferencesystem 26 containing the cluster of destination conference functions 18.The conference system 26 may be arranged wherein the respectivedestination conference functions 18 may subscribe to or join a multicaststream that is terminated at the conference system 26. The destinationstream provided from the destination conference functions 18 to thecorresponding destination devices 14 may be a unicast stream. Asillustrated, the source mixing information provided by the sourceconference function 16 represents the control information that isassociated with the multicast streams. Although shown separately, thesource mixing information is control information that may be providedin-band or out-of-band with respect to the respective multicast streamsto control selection and processing of media content. The mixingfunction 20 is not illustrated in FIG. 5.

Given the logical nature of the functionality of the present invention,the various functions may be implemented in various ways and at variouslocations. Select variations are provided in FIGS. 6A-6E; however, theseexamples are only a few of the possible variations of implementing thepresent invention. With particular reference to FIG. 6A, the destinationconference function 18 and the mixing function 20 may be provided in theconference system 26. In FIG. 6B, the destination conference function 18is provided in the conference system 26, while the mixing function 20 isprovided in a separate mixing system 28. In FIG. 6C, the destinationconference function 18 is implemented in the conference system 26, andthe mixing function 20 is implemented in the destination device 14. InFIG. 6D, the destination conference function 18 and the mixing function20 are implemented in the destination device 14. As illustrated in FIG.6E, the conference system 26 may support any number of source conferencefunctions 16, destination conference functions 18, and mixing functions20.

The present invention is particularly applicable in conferencingenvironments requiring voice or audio-based conferencing components. Assuch, basic voice conferencing and multimedia conferencing requiringvoice conferencing benefit from the present invention. When themultimedia conferencing includes video conferencing or applicationsharing along with voice conferencing, video or application content topresent to conference participants may be selected based the audiocontent chosen to present to the conference participants. Alternatively,triggers provided in the control information may be used to select oneor more video streams as an active video stream. Depending on how thecontrol information is configured, different participants may receivedifferent video streams. For voice conferencing, additional audiocontent may be mixed with the active media streams to provide a morerich conference experience.

As illustrated in FIG. 7, an alternate media source 30, such as a gamingconsole, content server, or the like, is able to provide additionalmedia content, such as audio, graphics, video, or the like. Theadditional media content may be provided directly to the mixing function20, or may be passed through the destination conference function 18 andthen sent to the mixing function 20 along with the active media streamsto create the destination stream, which is sent to the destinationdevice 14. The dashed lines represent the various paths from thealternate media source 30 to the mixing function 20 for the additionalmedia content. The destination conference function 18 may facilitatesuch operation based on control information maintained at thedestination conference function 18, source mixing information receivedfrom the source conference, functions 16, or from the control node 24.Those skilled in the art will recognize variations in allowing thedestination conference function 18 to effectively control whatadditional media content is mixed with the active media streams, and howthe additional media content is processed for mixing.

Since the present invention allows a single participant to engage indifferent conferences, which may include the same or differentparticipants, the source conference function 16, destination conferencefunction 18, or both may need to keep track of the participantsassociated with each conference. In particular, the source anddestination conference functions 16, 18 may keep track of the respectivesource and destination conference functions 16, 18 associated with therespective participants. A table, such as that illustrated in FIG. 8,may be maintained at the source or destination conference functions 16,18 as well as the control node 24. As illustrated, three conferences, X,Y, and Z, are provided. Conference X is a conference involving each ofparticipants A-E, while Conference Y only includes participants A and B,and Conference Z only includes participants C, D, and E. In such ascenario, media content for Conference Y is only exchanged betweenparticipants A and B. Media content for Conference Z is only exchangedamong participants C, D, and E. Media content for Conference X isexchanged among all of the participants A-E. The control information maybe used to determine when media content from a particular participantshould be associated with the respective conferences. This may becomeimportant because the media content from any given source device 12 isthe same, in most scenarios.

The invention is also particularly applicable to virtual conferencingenvironments, including multiplayer gaming environments. For a givengame, simultaneous conferences may be established where each conferenceincludes a different collection of participants. For example, all of thegaming participants may be associated with one conference and separateteams formed from different groups of gaming participants are associatedwith additional conferences. In this example, participants on the sameteam may be able to hear each other regardless of their locations,whereas participants on opposing teams may only hear each other based ontheir relative proximity to one another, direction they are facing,gaming situation, gaming audio, and the like. Control information andadditional audio may be provided to the source or destination conferencefunctions 16, 18 to control what each participant can hear and how theyshould hear it in light of the different conferences and in-gameenvironment. The video or graphics position, angle, or the like may becontrolled in a similar way. As such, each participant receives a uniqueconference and gaming experience, which may entail receiving mediacontent from multiple conferences at the same time.

Turning now to FIG. 9, a virtual conference environment is illustratedas having five participants, A-E, located at five locations, L0-L4,respectively. Assuming the virtual conference environment is illustratedfrom the perspective of participant E, who is located at location L0,audio content provided to participant E from participants A-D may behandled differently depending on the type of conference environment.Regardless of the type of conference environment, assume thatparticipant A is located to the northwest of participant E at locationL1. Participant A is sufficiently close to participant E such that thereis no natural attenuation of participant A's audio from the perspectiveof participant E. Participant B is located to the southwest ofparticipant E, at location L2. The distance between participant B andparticipant E is sufficient that audio from participant B is attenuated−3 dB with respect to participant E. Participant C is located to thesoutheast of participant E, at location L3. The distance betweenparticipant C and participant E is sufficient that audio fromparticipant C is attenuated −6 dB with respect to participant EParticipant D is located to the east of participant E, at location L4.The distance between participant D and participant E is sufficient thataudio from participant D is attenuated −10 dB with respect toparticipant E.

Based on the above, in a traditional conference environment, the audiocontent presented to participant E from participants AD may be processedto amplify the different audio content such that participant E receivesthe same relative volume level from the respective participants A-D. Assuch, the audio content from participant A is provided to participant Eat a nominal volume level. The volume level for the audio content fromparticipant B is increased by 3 dB, the volume level of audio contentfrom participant C is increased by 6 dB, and the volume level for theaudio content from participant D is increased by 10 db.

In an alternative conference environment, such as a gaming environment,the respective participants may be able to talk to one another, assumingthey are participating in one or more conferences. Although thedifferent players may be talking at roughly the same level, theirvirtual voice levels may be decreased based on the distance between theparticipants. With the example provided in FIG. 9, the volume levelassociated with the audio content from participant A is not decreased;however, the volume level for the audio content from participants B, C,and D is reduced by −3 dB, −6 dB, and −10 dB, respectively. As such,audio content provided to participant E from the other participants A-Dis processed based on the relative distance the respective participantsare from each other. Further processing may be provided to take intoconsideration the direction each of the participants is facing withrespect to one another, background noises associated with their virtuallocations, and the like. The audio content from the respectiveparticipants may be encoded for surround sound, such that a givenparticipant may be able to automatically determine whether anotherparticipant is in front of them, behind them, or beside them, based onhow the audio content is encoded and presented to that particularparticipant.

With reference to FIGS. 8 and 9, an example of how the variousparticipants may be arranged in conference within a gaming environmentis provided. Initially, assume that all of the participants areassociated with Conference X. Participants A and B are associated withConference Y, and participants C-E are associated with Conference Z. Assuch, participants A and B may be on a first team, and participants C-Emay be on a second team. In a first person shooter type game,participants A and B on the first team may have in-game communicationheadsets that allow them to talk to one another regardless of theirrespective positions. The respective source and destination conferencefunctions 16, 18 for the various participants will cooperate to allowparticipants A and B to always communicate with each other regardless oflocation, as well as allow participants C-E to always communicate witheach other regardless of position. Participants on opposing teams areonly able to communicate with one another if they are within a givenproximity of one another.

As such, Conferences Y and Z always facilitate intra-conferencecommunications, whereas Conference X facilitates inter-conferencecommunications. Conferences Y and Z always allow the team members tocommunicate with one another, whereas Conference X is established suchthat only select members from either team are available to communicatewith each other depending on their relative proximity. Thus, ifparticipant B and participant C are within a given proximity of oneanother, but no other participants are close to participants B and C,only participants B and C can communicate with one another overConference X. Although the conferences are always active, the variouscontrol information is employed to only select the media content forparticipants B and C for exchange in the above situation. The mediacontent provided by participants A, D, and E are not selected, and thusare not exchanged over Conference X in this example.

In this example, participant B may be able to receive media content fromparticipant C via Conference X, and from participant A via Conference Y.Since the destination conference function 18 is receiving media contentfrom all participants, in light of conferences X, Y, and Z, the variousmedia content from the different conferences may be selected asappropriate and mixed together to form a single destination stream topresent to the corresponding destination device 14 of participant B.This is the case because participant B should be able to hearparticipant C due to their proximity, and participant A due to theirin-game communication equipment. To further the example, supplementalaudio content may be selected based on the relative locations ofparticipants B and C and mixed into the destination stream, such thatparticipant B is able to hear participant A, participant C, and theappropriate background sounds in light of her location. Since theseconferences are controllable on an individual basis by all of theparticipants, each participant can give precedence to herself or toothers within the respective conferences. If a team leader is talking,all the other team members may receive the team leader's comments evenif all the team members are talking at the same time. Within any givenconference setting, the number of incoming media streams to mix into adestination stream may be limited to a select number, where certainmedia streams are given priority over others. In normal operation, themedia streams having the highest relative volume levels may be selectedfor the destination stream, unless the team leader is speaking. Thoseskilled in the art will recognize the tremendous flexibility inarranging conferences as well as the significant utility imparted by theinvention.

With reference to FIG. 10, an exemplary table is provided to illustratehow certain control information may be provided at a destinationconference function 18 for participant E, who is located at location L0.In this example, the conference table identifies the source conferencefunctions 16 for participants A-D, their relative priorities in generalor at any given time, orientation information, and any supplementalmedia information. The orientation information may include the actual orrelative locations L1-L4, as well as their positions and influence onaudio levels. The supplemental media information may identify additionalmedia content to mix into the destination stream to provide soundeffects and the like. Additional processing information may be providedto not only identify what background audio to provide, but a relativelevel at which to provide the background audio content. Similar tablescould be provided at the source conference function 16.

Notably, the present invention is particularly applicable in audio orvoice-based conferencing, but is readily applicable to other types ofmedia, such as text, video, and graphics, or any combination thereof. Assuch, the conferences may be established for any single type of mediacontent or a combination of different types of media content. Thedifferent types of media content may be integrated into a common mediastream or provided in parallel media streams. Those skilled in the artwill recognize numerous ways for conference participants to configuretheir respective source and destination conference functions 16, 18 tocustomize their conference experience on a participant-by-participantbasis. Additionally, the present invention is equally applicable tovoice only conferences, which are those that only include voice.

FIG. 11 is a block representation of a conference system 26. Theconference system 26 may include a control system 32 having sufficientmemory 34 for the requisite software 36 to operate as described above.Depending on how the conference system 26 is configured, the software 36may support any number of source conference functions 16, destinationconference functions 18, and mixing functions 20. Notably, theconference system 26 may only allocate mixing functions 20 for thosedestination devices that do not include internal mixing functions 20 orare associated with some other mixing function 20. As such, theconference system 26 may provide mixing functions 20 internally forselect destination devices 14, and not for others. The control system 32may also be associated with one or more communication interfaces 38 tofacilitate communications with various entities using packet-basedcommunications.

FIG. 12 illustrates a user terminal 40, which may represent a sourcedevice 12 and a destination device 14, alone or in combination. The userterminal 40 will include a control system 42 with sufficient memory 44for the requisite software 46 to operate as described above. Inparticular, the software 46 may include source conference functions 16,destination conference functions 18, as well as mixing functions 20 inselect embodiments. All of these functions are not necessarilyimplemented in each user terminal 40, but may be selectively andindividually employed in any given user terminal 40. The control system42 may be associated with one or more communication interfaces 48 tofacilitate packet-based communications with various entities. Thecontrol system 42 may also be associated with a user interface 50, whichfacilitates presentation of various media to a user, and facilitates thereception of media and other control information from the user.

FIG. 13 provides a block representation of a control node 24 accordingto one embodiment of the present invention. The control node 24 willinclude a control system 52 having sufficient memory 54 for the software56 necessary to operate as described above. The software 56 may providean overall control function 58 to facilitate control of any number ofsource conference functions 16, destination conference functions 18,mixing functions 20, or the like. The control system 52 is alsoassociated with one or more communication interfaces 60 to facilitatepacket-based communications with various entities.

Those skilled in the art will recognize improvements and modificationsto the preferred embodiments of the present invention, All suchimprovements and modifications are considered within the scope of theconcepts disclosed herein and the claims that follow.

What is claimed is:
 1. A system comprising: at least one communicationinterface for coupling to a communication network; at least oneprocessor coupled to the communication interface; and at least onestorage element, storing instructions for execution by the at least oneprocessor, the instructions comprising: instructions executable toreceive media content from a plurality of media sources; instructionsexecutable to obtain control information associated with the receivedmedia content; and instructions executable to select media content fromthe received media content for presentation to at least one mediadestination in association with at least one media conference based onthe control information.
 2. The system of claim 1, wherein theinstructions comprise instructions executable to mix the selected mediacontent for presentation to the at least one media destination inassociation with the at least one media conference based on the controlinformation.
 3. The system of claim 2, wherein the at least onecommunication interface, the at least one processor and the at least onestorage element are incorporated in a conference system network element.4. The system of claim 3, wherein the instructions executable to selectmedia content from the received media content comprise instructionsexecutable to select respective media content for each of a plurality ofmedia destinations in association with the at least one mediaconference.
 5. The system of claim 4, wherein: the instructionsexecutable to select media content from the received media contentcomprise instructions executable to select respective media content foreach of a plurality of media destinations in association with the atleast one media conference; and the instructions executable to mix theselected media content comprise instructions executable to mixrespective selected media content for presentation to each of theplurality of media destinations in association with the at least onemedia conference.
 6. The system of claim 3, wherein the at least onecommunication interface, the at least one processor and the at least onestorage element are incorporated in a destination communication device.7. The system of claim 2, wherein: the at least one processor comprisesa destination conference function processor and a mixing functionprocessor; the at least one storage element comprises a destinationconference function storage element storing the instructions executableto select the media content and a mixing function storage elementstoring the instructions executable to mix the selected media content;and the at least one communication interface comprises at least onecommunication interface coupled to the destination conference functionprocessor and at least one communication interface coupled to the mixingfunction processor such that the destination conference functionprocessor can communicate with the mixing function processor via thecommunication network.
 8. The system of claim 7, wherein the mixingfunction processor, the mixing function storage element and the at leastone communication interface coupled to the mixing function processor areincorporated in a destination communication device.
 9. The system ofclaim 2, wherein the instructions comprise: instructions executable toreceive media content from at least one media source; and instructionsexecutable to generate the control information for association with thereceived media content.
 10. The system of claim 9, wherein the at leastone communication interface, the at least one processor and the at leastone storage element are incorporated in a conference system networkelement.
 11. The system of claim 9, wherein: the at least one processorcomprises a plurality of source conference function processors, eachsource conference function processor incorporated in a respective sourcecommunication device; the at least one storage element comprises aplurality of source conference function storage elements, each sourceconference function storage element incorporated in a respective sourcecommunication device and storing respective instructions executable togenerate respective media content and respective instructions executableto generate respective control information for association with therespective received media content; and the at least one communicationinterface comprises at least one respective communication interfacecoupled to each source conference function processor such that eachsource conference function processor can communicate via thecommunication network with at least one other processor configured toexecute the instructions executable to select media content forpresentation to the at least one media destination in association withat least one media conference.
 12. The system of claim 11, wherein theeach source conference function processor sends respective controlinformation for association with its respective received media contentto each processor of the at least one other processor for use of therespective control information in selection of media content forpresentation to the at least one media destination.
 13. The system ofclaim 12, comprising a control node, wherein source conference functionprocessors send control information to the at least one other processorvia the control node.
 14. The system of claim 6, wherein the destinationcommunication device is configured to perform source conferencefunctions for media content generated at the destination communicationdevice such that the destination communication is operable as both asource communication device and a destination communication device. 15.The system of claim 2, comprising at least one additional media source,each additional media source configured to provide respective additionalmedia content wherein the instructions executable to mix the selectedmedia content for presentation to the at least one media destination inassociation with the at least one media, conference compriseinstructions to mix the addition media content with selected mediacontent for presentation to the at least one media destination.
 16. Thesystem of claim 1, wherein the instructions executable to select mediacontent from the received media content for presentation to at least onemedia destination in association with at least one media conferencecomprise instructions executable to select media content forpresentation to at least one media destination in association withmultiple simultaneous media conferences.
 17. The system of claim 16,wherein the instructions comprise instructions executable to mix theselected media content for presentation to the at least one mediadestination in association with the multiple simultaneous mediaconferences.
 18. The system of claim 17, wherein the at least one mediaconference presents a virtual conference environment at the at least onemedia destination.
 19. The system of claim 18, wherein the instructionsexecutable to mix the selected media content for presentation to the atleast one media destination in association with the at least one mediaconference comprise instructions to process media content to simulaterelative locations of conference participants.
 20. The system of claim18, wherein the at least one media conference presents a virtualconference environment for at least one electronic game.
 21. The systemof claim 1, wherein the media content comprises audio content.
 22. Thesystem of claim 1, wherein the media content comprises video content.23. The system of claim 1, wherein the media content comprises textcontent.
 24. The system of claim 1, wherein the media content comprisesgraphics content.
 25. The system of claim 1, wherein the media contentcomprises at least two of audio content, video content, text content andgraphics content.
 26. The system of claim 1, wherein the instructionscomprise instructions executable to integrate selected media contentfrom a plurality of sources into a common media stream for presentationto the at least one media destination.
 27. The system of claim 1,wherein the instructions comprise instructions executable to provideselected media content from a plurality of sources in parallel mediastreams for presentation to the at least one media destination.
 28. Thesystem of claim 1, wherein the control information is received in-bandwith the media content.
 29. The system of claim 1, wherein the controlinformation is received out-of-band separately from the media content.30. The system of claim 9, wherein the instructions executable togenerate the control information for association with the received mediacontent comprise: instructions executable to obtain participantinformation for each media conference of the at least one mediaconference; and instructions executable to identify at least one mediadestination for the received media content based on the participantinformation.
 31. The system of claim 30, wherein the instructionsexecutable to generate the control information for association with thereceived media comprise instructions executable to associate mediacontent received from one media source with multiple simultaneous mediaconferences.
 32. The system of claim 9, wherein the instructionsexecutable to generate the control information for association with thereceived media content comprise: instructions executable to obtainpriority information for each media conference of the at least one mediaconference; and instructions executable to generate control informationcomprising source mixing information based on at least the priorityinformation for each media conference of the at least one mediaconference.
 33. The system of claim 32, wherein the priority informationbears on at least one relative priority for presentation of the receivedmedia content to at least one media destination participating in the atleast one media conference.
 34. The system of claim 9, wherein theinstructions executable to generate the control information forassociation with the received media content comprise: instructionsexecutable to obtain orientation information for each media conferenceof the at least one media conference; and instructions executable togenerate control information comprising source mixing information basedon at least the orientation information for each media conference of theat least one media conference.
 35. The system of claim 34, wherein theorientation information identifies a location to be associated with thereceived media content.
 36. The system of claim 35, wherein, theorientation information identifies the location relative to other mediacontent sources associated with the at least one media conference. 37.The system of claim 34, wherein the orientation information identifies adirection to be associated with the received media content.
 38. Thesystem of claim 37, wherein the orientation information identifies thedirection relative to other media content sources associated with the atleast one media conference.
 39. The system of claim 9, wherein theinstructions executable to generate the control information forassociation with the received media content comprise: instructionsexecutable to obtain supplemental information for each media conferenceof the at least one media conference; and instructions executable togenerate control information comprising source mixing information basedon at least the supplemental information for each media conference ofthe at least one media conference.
 40. The system of claim 39, whereinthe supplemental information relates to additional media content to beprovided to at least one media destination associated with the at leastone media conference.
 41. The system of claim 40, wherein thesupplemental information relates to at least one of background sounds,music, graphics and video clips.
 42. The system of claim 9, wherein theinstructions executable to generate the control information forassociation with the received media content comprise instructions togenerate the control information once per media conference.
 43. Thesystem of claim 9, wherein the instructions executable to generate thecontrol information for association with the received media contentcomprise instructions executable to generate initial control informationfor a media conference for use when the media conference begins.
 44. Thesystem of claim 43, wherein the instructions executable to generate thecontrol information for association with the received media contentcomprise instructions to generate updated control information for themedia conference for use during the media conference.
 45. The system ofclaim 44, wherein the updated control information is a subset of theinitial control information.
 46. The system of claim 4, wherein theinstructions executable to select media content from the received mediacontent for presentation to at least one media destination inassociation with at least one media conference comprise instructionsexecutable to process the received media content to determine respectiveacoustic properties for media content received from each respectivemedia source.
 47. The system of claim 46, wherein the respectiveacoustic properties indicate respective audio levels.
 48. The system ofclaim 4, wherein the instructions executable to select media contentfrom the received media content for presentation to at least one mediadestination in association with at least one media conference comprise:instructions executable to obtain priority information for each mediaconference of the at least one media conference; and instructionsexecutable to select the media content for presentation to the at leastone media destination based at least in part or the priorityinformation.
 49. The system of claim 5, wherein: the instructionsexecutable to obtain control information associated with the receivedmedia content comprise instructions executable to obtain priorityinformation for each media conference of the at least one mediaconference; and the instructions executable to mix the selected mediacontent comprise instructions executable to mix respective selectedmedia content for presentation to each of the plurality of mediadestinations based at least in part on the priority information.
 50. Thesystem of claim 5, wherein: the instructions executable to obtaincontrol information associated with the received media content compriseinstructions executable to obtain orientation information for each mediaconference of the at least one media conference; and the instructionsexecutable to mix the selected media content comprise instructionsexecutable to mix respective selected media content for presentation toeach of the plurality of media destinations based at least in part onthe orientation information.
 51. The system of claim 50, wherein theorientation information identifies a location to be associated with thereceived media content.
 52. The system of claim 51, wherein theorientation information identifies the location relative to other mediacontent sources associated with the at least one media conference. 53.The system of claim 51, wherein the orientation information identifies adirection to be associated with the received media content.
 54. Thesystem of claim 53, wherein the orientation information identifies thedirection relative to other media content sources associated with the atleast one media conference.
 55. The system of claim 5, wherein: theinstructions executable to obtain control information for associationwith the received media content comprise instructions executable toobtain supplemental information for each media conference of the atleast one media conference; and the instructions executable to mix theselected media content comprise instructions executable to mixrespective selected media content for presentation to each of theplurality of media destinations based at least in part on thesupplemental information.
 56. The system of claim 55, wherein thesupplemental information relates to additional media content to beprovided to at least one media destination associated with the at leastone media conference.
 57. The system of claim 56, wherein thesupplemental information relates to at least one of background sounds,music, graphics and video clips.